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Site visits provide 
authorizers with 
a mechanism for 
verifying and 
corroborating 
information 
collected through 
reports , gauging 
the culture and 
climate of a 
school , gathering 
evidence of 
performance 
from a range of 
perspectives , and 
demonstrating 
the commitment 
of the authorizer 
to authentic 
accountability. 


The charter school concept is built on the 
premise that in return for freedom, charter 
schools are held accountable to promises 

made in their charter. It is the autho- 
rizer’s role to ensure that charter 
schools uphold their end of the 
agreement by providing appropriate 
oversight of each charter school. 


The term “oversight” sounds simple 
enough. Those performing the task, 
however, are acutely aware of its 
complexity. Monitoring and evaluating 
whether a charter school is in compli- 
ance with statutory and regulatory 
requirements and performing finan- 
cially, organizationally, and academi- 
cally requires the authorizer to 
examine a host of information about 
an individual school. Some of the this 
information is reported by the school 
itself; vehicles such as self-reports, 
testing data, and financial audits detail 
indicators of the academic and organi- 
zational health of a school. While 
such documentation gives important 
information on how a school is 
performing, many authorizers have 
found that “seeing is believing” and 
have incorporated on-site reviews, or 
site visits, into their charter school 
oversight process. 


The benefits of on-site reviews are many. 

Site visits provide authorizers with a mecha- 
nism for verifying and corroborating informa- 
tion collected through reports, gauging the 
culture and climate of a school, gathering 
evidence of performance from a range of 
perspectives, and demonstrating the commit- 
ment of the authorizer to authentic accounta- 
bility. To achieve these purposes, a site visit 
must be much more than a passing drop-in 
or a compliance check. Rather, a quality site 
visit takes a holistic look at the school to 
determine how it is performing academically 
and organizationally and the extent to which 
it is serving the students who are enrolled. 

There are a variety of models of how on-site 
reviews may be conducted and how they fit 
into the larger system of authorizer oversight. 
This Issue Brief will examine the key 
components of a quality on-site review as 
well as discuss choices to consider when 
designing school visits into a charter school 
oversight system. 

The Core Components of a 
Quality On-site Review 

A few key principles are central to planning 
and executing a quality site visit. These 
include creating a protocol to articulate the 
purpose and expectations for the visit; find- 
ing and training appropriate visitors; gather- 
ing data from a wide array of stakeholders; 
basing all findings and judgments on clear 
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evidence; and documenting this evidence in 
a report for the benefit of the school, the 
authorizer, and the public. 

Clear Protocol and Expectations 

Authorizes will find that clearly articulating 
both the purpose of the on-site visit and the 
expectations for what a site visit will look 
like and what type of evidence they will seek 
helps both the visiting team and the school. 
This is best done through issuing a protocol 
that is consistent for all site visits and sets 
forth the purpose and expectations for the 
visit in terms of both process and substance — 
the structure of the visit and the criteria/ques- 
tions that will guide the time at the school. A 
site visit protocol may include the following: 

Purpose of the Visit: A protocol should 
clearly explain the purpose of the site visit, 
how it fits into the larger system of oversight, 
and what the final outcome of the visit will 
be. For example, a school may view a visit by 
its authorizer as punitive in nature rather than 
a key component of charter school accounta- 
bility. Understanding the purpose of the visit 
is important to setting a productive tone for 
both the school and visiting team. 

Guiding Criteria or Questions: The criteria 
for each visit should be clear to the school 
and the public. What is the authorizer ulti- 
mately looking to find when visiting a 
school? What are the key questions that the 
team will attempt to answer? What are the 
criteria and indicators by which the school 
will be measured on this review? If a school 
understands the standards to which an 
authorizer is holding it, then it can gather 
and present data that show its strengths in 
these areas. 


Defining the criteria also helps 
focus team members on what is 
most important. Schools are 
complex organizations. If team 
members do not have a clear 
understanding of the evidence they 
are looking for when they enter the 
building, there is the potential for 
visitors to get bogged down in 
each and every aspect of the 
school. As a result, the team will 
have less than complete informa- 
tion at the end of the visit. 

Schedule/Expected Interviews: 

A schedule provides a general 
framework for how time at the 
school will be spent (see Example 1). 

This helps direct the visit and allows for 
better planning. For example, the protocol 
should identify which stakeholders will be 
interviewed during the visit. This information 
may be especially important if interviewees 
include members who are not employed in 
the school building, such as parents, 

Trustees, or community partners. While the 
actual schedule will be particular to each 
school’s start time, programmatic offerings, 
and class schedules, the sample schedule 
gives the school an idea in advance of what 
the visit will hold for its community. 

Requested School Documents: The visiting 
team may need to review documents, either 
before or during the visit, that are not readily 
available in the authorizer’s files. Such docu- 
ments may include class schedules, curricular 
documents, teacher rosters, or professional 
development plans. A protocol that lists 
document requests gives the school ample 
time to gather and arrange data for the team. 


The criteria for each 
visit should be clear 
to the school and 
the public. What is 
the authorizer 
ultimately looking to 
find when visiting a 
school? What are 
the key questions 
that the team will 
attempt to answer? 
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Sample Interview Questions: It is helpful for 
both the school and team members to know 
the types of questions visitors will be asking 
(see Example 2). Team members can use 
sample questions to focus interviews on the 
standards of the visit. In a school where 
there are multiple interviews happening 
simultaneously, teams can choose to pick a 
few key questions from the list to ask each 
constituent so that consistent evidence can 
be gathered from each person. 

Visitor Expectations: Authorizes should 
consider including additional elements in a 
site visit protocol to the benefit of all partici- 
pants: a Code of Conduct defining the 


expected behavior and tone of the visiting 
members, a timeline of steps in the process 
such as when documents are due or when 
the final schedule should be set, or 
frequently asked questions covering such 
issues as the duties of a site visitor and the 
charter school. 

A protocol serves many purposes, but one 
of its most functional roles is to help all 
parties understand the process. Schools 
are well aware that the visit may have real 
consequences. Few schools will approach 
a visit by their authorizer without some 
apprehension. Clarity and transparency 
will help dispel some of this anxiety. 


Example 1-Schedule: The Massachusetts Department of Education’s Charter School Office provides a sample 
schedule in its site visit protocol that gives a general guideline of how its one-day reviews, conducted in a school’s 
second and third year of operation, will proceed. The schedule includes time blocks for meetings with a school’s 
governing board and administration; focus groups with parents, students, and teachers; classroom visits; and 
document review. The schedule provides for the collection of data from a wide array of sources and is flexible enough 
to be changed according to each school’s particular needs. Following is the sample schedule: 


Time 

Team Member A 

Team Member B 

Team Member C 

Team Member D 

Team Member E 

7:30 a.m. 

Team Meeting and Orientation 

8:00-9:00 

Meet with Board of Trustees 

9:00-10:00 

Meet with Administration 

10:00-11:15 

Classroom Visits 

Classroom Visits 

Classroom Visits 

Classroom Visits 

Classroom Visits 

11:15-12:00 

Student Focus 
Group 

Student Focus 
Group 

Class Visits 

Parent Focus 
Group 

Parent Focus 
Group 

12:00-1:00 

Lunch with Teachers 

1:00-1:30 

Initial Team Discussion 

1:30-2:30 

Additional Fact 
Finding 

Additional Fact 
Finding 

Additional Fact 
Finding 

Document Review 

Document Review 

2:30-4:00 

Team Work Time 

4:00-4:30 

Presentation of general observations and findings to school leaders 


Example 2-Sample Interview Questions: The Mayor of Indianapolis authorizes schools in the city of Indianapolis. 

As part of its oversight of schools, the mayor’s office conducts visits throughout the life of the charter. In its Charter 
School Site Visit Protocol, team members are given a list of relevant and useful questions that may be used in each 
focus group during the day. There are sample questions for Board members, parents, students, teachers, the executive 
director or head of school, the principal or instructional leader, curriculum specialists, and business manager. Sample 
questions are provided for the different areas of inquiry, both organizational and academic, and are specific to the 
group being interviewed. You can view the Charter School Site Visit Protocol at: 
http://www6.indygov.org/mayor/charter/accountability_report/pdf/expert_site_visit.pdf 




Pre-visit Document Review 

Site visit teams that review relevant 
documents, such as test scores, demographic 
information, annual reports, past site visit 
reports, charter application, and financial 
data prior to the visit will come into a school 
better prepared to carry out the purposes of 
the visit. This preparation gives context to 
the school and allows visitors to think of 
probing questions particular to this 
organization’s situation. One external review 
company has team members prepare for 
visits by outlining initial hypotheses and 
additional questions for each area of inquiry 
based on the documents that were provided . 1 
An appropriately prepared team will ask 
better questions and likely find more 
robust evidence. 

Choosing and Training Visitors 

Appropriately staffing visits is critical to the 
overall effectiveness of the site visit. 
Authorizers must first find visitors that have 
the expertise needed to find evidence for the 
different criteria. For instance, in visits that 
evaluate both the academic and organiza- 
tional program of the school, visitors with 
those varying areas of expertise should be 
included. While an organizational change 
consultant may know a fair amount about 
how school administration works, they may 
know very little about curriculum or teach- 
ing. Authorizers should also consider the 
particular mission or make-up of a school 
when choosing visitors, including demo- 
graphics and grade-level expertise. Mission 
elements matter as well: an elementary school 
focused on the integration of music and arts 
in the classroom may necessitate a visitor 
with experience in very different areas than 
a school sewing former out-of-school youth. 

A key visitor on an on-site review is the team 
leader. This person is in contact with the 
school leadership regarding scheduling prior 
to the visit, moves the conversations along 
according to the protocol and schedule, and 
is ultimately responsible that the information 


sought has been gathered. They act as the 
face of the team to the school and set the 
tone for the visit. Some authorizers use a 
member of their own staff to fill this role 
because he or she has a 
deep understanding of 
the protocol and purpose 
of the visit. Other autho- 
rizers use external 
consultants for this role 
and provide this position 
with additional training. 

Once visitors are chosen, 
authorizers should 
provide training on the 
process and protocol of 
the site visit. This train- 
ing adds to consistency 
in implementation of the 
protocol and helps focus 
each visitor on the particular criteria for the 
visit at hand. Training for visitors can take 
multiple forms ranging from full day trainings 
to a few hour conference call to a brief meet- 
ing before the start of the visit. While more 
extensive training is ideal, this is clearly a 
financial consideration for authorizers as 
trainings can be costly. 

Stakeholder Input 

Interviews are the crux of an on-site review 
and provide key pieces of data. Teachers, 
administrators, parents, governing board 
members, and students should, to the greatest 
extent possible, be involved in discussions 
during the visit. When appropriate, commu- 
nity partners or management companies 
should be included in 
the interview schedule as 
well. Involving each of 
these groups serves 
multiple purposes: it 
provides the team with a 
large pool of information, allows for verifica- 
tion of facts from different perspectives, and 
creates an environment where the whole 
school community feels heard. 


Interviews are the crux of 
an on-site review and 
provide key pieces of data. 


Site visit teams that review 
relevant documents , such 
as test scores , demographic 
information , annual reports , 
past site visit reports , 
charter application , and 
financial data prior to the 
visit will come into a school 
better prepared to carry out 
the purposes of the visit. 


Structuring interviews is a critical chore for 
the site visit team. In many schools, the size 
of the faculty makes it difficult to talk with 
each individual, so the team must strategi- 
cally choose which members of the faculty to 
interview, covering a range of grade levels 
and teacher longevity at the school. In addi- 
tion, team members should communicate a 
high level of confidentiality to interviewees 
so that they feel free to speak openly in a 
confidential environment. 


Report of Findings 

The team’s findings should serve as the basis 
for a written report. Authorizers use different 
techniques for site visit report writing, ranging 
from bulleted lists of findings to a full narra- 
tive. Depending on the chosen method, some 
teams may leave the school with a final report 
written, while others leave with findings that 
the designated team writer later fleshes out 
with supporting evidence and narrative. 


Evidence-Based Findings 

All conclusions from the visit must be based 
on evidence collected and corroborated 
throughout the time at the school. “Evidence- 
based decision-making” is arguably the most 
important element of an on-site review. “The 
visiting team must build a base of evidence 
for each of its findings that would reasonably 
lead another set of individuals to come to a 
similar judgment of the school.” 2 

In order to present the most accurate and 
complete picture of the school, visitors need 
to enter the building committed to the practice 
of evidence-based decision-making. The team 
must focus on gathering enough evidence 
from enough different sources so that consis- 
tent verification outweighs any outlying 
comment. This approach also helps check any 
sort of bias that an individual team member 
might unwittingly bring into the building. 

Many authorizers require that the site visit 
team finalize all of the major findings prior to 
the end of the visit. This requirement supports 
a consensus model of decision-making and 
helps ensure that findings have sufficient 
evidence and that team members agree. 


As the final report documents important 
information for an authorizer’s oversight of 
a charter school, it is good to provide an 
opportunity for schools to respond to the 
team’s findings in writing. Some authorizers 
require a written response or a plan be 
submitted directly related to the team’s 
conclusions while others let schools choose 
whether they will respond. 

Authorizers should consider the accessibility 
of such reports in terms of the school 
community and public (see Example 3). 
Sharing the important findings with the 
school’s stakeholders and larger community 
supports charter school transparency 
and accountability. 

Variables and Options 

In designing and implementing quality site 
visits, there are a host of other issues for an 
authorizer to consider in relation to their 
needs and oversight philosophy. These 
include decisions around when visits will 
take place, whether to use paid visitors, and 
whether authorizers should be participants 
on each of these visits. 


Example 3-Report of Findings: Official site visit reports help 
increase accountability. The important information collected during 
the day is documented and available throughout the life of the 
school. Some authorizers have made the findings from on-site 
reviews available and easily accessible to the public. The State 
University of New York’s Charter School Institute posts each of its 
Charter School Visit Reports online. This increases both the school 
and the authorizer’s transparency. To view these reports, visit 
www.newyorkcharters.org/charterny/institutereports.asp 


Length of Visit and Frequency 

The choice in the length of a visit depends 
on the depth of data expected to be 
collected. Typically, the more time spent at a 
school, the more extensive and complete the 
information is. 

Current authorizers use a variety of lengths, 
ranging from one to four days, and many 



combine longer and shorter visits throughout 
the life of the charter. A combination of visits 
allows for the collection of a wide range of 
information but keeps the process reasonable 
for both the school and the authorize!'. 

The frequency of visits can be an important 
factor in determining length. If an authorizer 
conducts visits annually, each visit can follow 
up on the previous one, making a one day 
visit more feasible. If on-site reviews are 
conducted only once over the term of the 
charter, a longer review is probably needed. 

Volunteers and Consultants 

Good visitors are a critical part an on-site 
review. For some authorizers, this means 
hiring independent consultants or a company 
in the business of conducting school evalua- 
tions. There are some clear benefits to hiring 
consultants or contracting with a company, 
including freeing up some of the authorizer’s 
time, allowing for an external perspective, 
and gaining the experience and infrastructure 
of a company who can hire and train strong 
team members. However, it is difficult for 
some authorizers to find the funds to pay for 
these services. 

In response, some authorizers have chosen 
to use volunteers (see Example 4), trained 
and led by either a paid consultant or a 
member of the authorizer staff. In return, 
visitors are offered a chance to get an in- 
depth look at what a school is doing, under- 
stand more fully the process of accountability 
for charter schools, or even receive profes- 
sional development points/ continuing 
education credits that can be used toward 
re-certification. Authorizers can attract volun- 
teers from a wide pool including areas of 
unique expertise beyond just those of evalua- 


Example 4: The Colorado League of Charter Schools (League), 
while not a chartering entity, works with districts throughout 
Colorado who do authorize charter schools to help them implement 
a rigorous and useful accountability system. This includes a two 
and a half day on-site visit conducted at the school in its third or 
fourth year of operation. The visit is conducted as a peer review, 
staffed by teachers, school leaders, and business managers. While 
the team leader is given a stipend, the rest of the visitors are 
volunteers. Those who are interested or eligible receive continuing 
education credits and when one of the volunteers is a teacher, the 
League provides substitute pay to that individual’s school. Travel, 
food and lodging are provided for the whole team. To find out more, 
visit www.coloradoleague.org/accountability_programs.html 


tion or educational consultants. There are 
drawbacks to using volunteers, including the 
limit it puts on the time one can expect them 
to give. 

Authorizer participation 

The participation of the authorizer on a visit 
varies between chartering entities. Some 
authorizers participate in all visits (see 
Example 5), allowing them to ensure that the 
protocol is adhered to as well as get a first 
hand look at the performance of the school. 
This participation ranges from monitoring the 
team in their deliberations to an active 
involvement in the interviews and generation 
of findings. On the other hand, some autho- 
rizers use a completely external review 
component in their oversight process to 
ensure a varied perspective on the school. 

Formative and Summative Evaluations 

Whether an evaluation is formative or 
summative is a matter of both purpose and 
timing. Formative evaluation is a method of 
judging a program even as its activities are 
still forming while summative evaluation 


Example 5: On-site reviews are an important part of the oversight functions of the District of Columbia Public Charter 
School Board (DCPCSB). Visits to DCPCSB authorized schools occur yearly and the visitors, including the team leader, 
are made up of external consultants that receive training from members of the DCPCSB staff. However, different than 
external review processes for other authorizers, each visit to these DC charter schools includes the participation of a 
member of the authorizer staff. This person’s role is to monitor the site visit process to ensure that it goes according 
to the purpose and protocol of the visit. 




focused exclusively on evaluating the school 
according to the set questions/criteria or 
whether the visit will include a component of 
technical assistance in the form of recom- 
mendations. On-site reviews can be strictly 
evaluative and team members are instructed 
to avoid giving advice or recommendations 
to the school. On the flip side, the team 
could make pointed suggestions and recom- 
mendations based on its findings during the 
visit. An authorizer must decide whether the 
positives of this practice, which gives the 
school concrete advice about next steps from 
those knowledgeable in the field, outweigh 
the potential drawbacks, including teams 
giving bad or self-interested advice and/or 
blurring the lines between the role of over- 
sight and technical assistance. 

Conclusion 

An authorizer must develop and implement a 
school review process that works best for its 
schools and purposes. This Issue Brief identi- 
fies the major elements for authorizes to 
consider when designing site visits into a 
comprehensive oversight system. Clearly, the 
cost in terms of time, money and personnel 
is real and present in the minds of all author- 
izing agencies and a serious consideration for 
how on-site reviews can be used. That said, 
on-site reviews can give the authorizer rich 
information not available through documents 


Example 6: The Charter School Office at the Massachusetts Department of Education (CSO) and the Charter Schools 
Institute at the State University of New York (CSI) have very similar processes for overseeing the charter schools they 
authorize. This includes an accountability system that uses both authorizer led reviews throughout the charter coupled 
with a review performed by an external evaluation team. A key difference, however, is the timing of the external visit, and 
therefore the extent to which it is a formative or summative evaluation. Massachusetts charter schools are visited by the 
external evaluation team once the school has applied for renewal of its charter, usually in year five. While the second and 
third year one-day visits conducted by the CSO are more formative in nature, these external visits are purposed to provide 
the most complete and up to date information about the school and its program at the time of renewal. 

CSI has structured their visits differently. Under its system, the external review is conducted during a school’s third 
year of operation. As stated in the accompanying reports, “[t]he visit provides an independent assessment of the 
school's progress and provides recommendations to the school as it prepares to apply for charter renewal in its fifth 
year of operation.” The team is charged to give the school recommendations, allowing the school to address and 
potentially remedy these problems prior to its application for renewal. At renewal time, the CSI team conducts a 
summative evaluation of the school’s program. 


focuses on assessing programs at the end of 
their activities. 3 In a much less technical 
sense, these two concepts factor an autho- 
rizer’s decision of when to perform on-site 
reviews in the life of a school’s charter (see 
Example 6). Some authorizers conduct their 
longest and most involved evaluations during 
the “adolescent years” of 
the charter. This 
approach gives a school 
detailed and rich 
feedback early in its 
existence, which it can 
use to focus its efforts. 
Other authorizers time 
the lengthiest visits at 
the end of the school’s 
charter to get as 
complete a picture as 
possible of the school’s 
performance at the time 
of the critical renewal 
decision. Timing of the 
visit plays an important 
role in how schools and 
authorizers are able to 
use the visits for their 
respective purposes. 

On a related point, another important deci- 
sion that an authorizer needs to make regard- 
ing an on-site review, whether internally or 
externally led, is whether the visit will be 


Another important decision 
that an authorizer needs to 
make regarding an on-site 
review , whether internally or 
externally led , is whether 
the visit will be focused 
exclusively on evaluating 
the school according to the 
set questions/criteria or 
whether the visit will include 
a component of technical 
assistance in the form of 
recommendations. 



and allow schools to know that accountabil- 
ity is real and active and that the authorizer 
is both serious and supportive in that role. 

About the Author 

Rebecca Wolf DiBiase is an independent 
consultant working with schools and non- 
profit organizations in the area of organiza- 
tional leadership and planning. She is a 
former middle school teacher and has been 
involved in education reform and charter 
schools in Massachusetts for seven years. 

Most recently, she was Director of 
Accountability in the Charter School Office of 
the Massachusetts Department of Education. 
She is currently pursuing her Masters in 
Public Administration at Harvard University’s 
Kennedy School of Government. 


References 

Visit the following websites to obtain 
additional information about the prac- 
tices highlighted throughout this brief. 

Colorado League of Charter Schools 
www. coloradoleague . org 

DC Public Charter School Board 
www. dcpubliccharter. com 

Massachusetts Department of 
Education, Charter School Office 
www. doe . mass . edu/ charter 

Mayor of Indianapolis (IN), 

Charter School Office 
www6 . indygov . org/ mayor/ 
charter/ index, htm 

State University of New York, 

Charter Schools Institute 
www . ne wyorkcharters . org 


Endnotes 

1 Pre-Visit Hypothesis Sheet, 
School Works LLC, 
www.schoolworks.org. 

2 Massachusetts Renewal 
Inspection Protocol, p.9, 
http://www.doe.mass.edu/ 
cha rter/ pdf/ protocol . pdf. 

3 Bhola, H. S. 

1990. Evaluating “Literacy 
for development" projects, 
programs and campaigns: 
Evaluation planning, design 
and implementation, and 
utilization of evaluation 
results. Hamburg, Germany: 
UNESCO Institute for 
Education; DSE [German 
Foundation for International 
Development], xii, 306 
pages, as quoted by SIL 
International. 


To further your learning, register now for NACSA’s 2004 Annual Conference October 25-26, 2004 in Philadelphia, Pennsylvania. 
Visit our website, www.charterauthorizers.org, for conference and registration details. 
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